Electric camera

ABSTRACT

An electric camera includes an image sensing device with a light receiving surface having N vertically arranged pixels and an arbitrary number of pixels arranged horizontally, N being equal to or more than three times the number of effective scanning lines M of a display screen of a television system, a driver to drive the image sensing device to vertically mix or cull signal charges accumulated in individual pixels of K pixels to produce, during a vertical effective scanning period of the television system, a number of lines of output signals which corresponds to 1/K the number of vertically arranged pixels N of the image sensing device, K being an integer larger than an integral part of a quotient of N divided by M, and a signal processing unit having a function of generating image signals by using the output signals of the image sensing device.

BACKGROUND OF THE INVENTION

[0001] The present invention relates to a photography related to videocameras, camcorders, digital still cameras and others using asolid-state image sensing device, and more particularly to an electriccamera using a solid-state image sensing device with a large number ofpixels.

[0002] Electric cameras using solid-state image sensors such as CCDs(charge-coupled devices) include a so-called video camera or camcorderfor taking moving images and a so-called digital still camera for takingstill images. In recent years, video cameras with a still image takingfunction and digital still cameras with a moving image taking functionhave become available.

[0003] In a video camera to photograph moving images, it is generallyassumed that the video is viewed on a display such as television monitorand thus the camera is designed to produce output signals conforming toa television system such as NTSC and PAL. Therefore, the effectivenumber of vertically arranged pixels or picture elements on the imagesensing device used in such a camera is determined to enable televisionsignals to be generated. The NTSC system, for example, performsinterlaced scanning on two fields, each of which has an effectivescanning line number of about 240 lines (the number of scanning linesactually displayed on the monitor which is equal to the number ofscanning lines in the vertical blanking period subtracted from the totalnumber of scanning lines in each field). To realize this, the imagesensing device has about 480 pixel rows as the standard effective numberof vertically arranged pixels. That is, the signals of two verticallyadjoining pixels in each field are mixed together inside or outside theimage sensing device to generate about 240 scanning lines, and thecombinations of pixels to be cyclically mixed together are changed fromone field to another to achieve the interlaced scanning.

[0004] Some image sensing devices to take moving images according to theNTSC system have an area of pixels for image stabilization added to thearea of effective pixel area, thus bringing the effective number ofvertically arranged pixels to about 480 or more. In this case, an areabeyond 480th pixels is read out at high speed during the verticalblanking period and therefore the signals thus read out are not used aseffective signals. Therefore, the video signals can only be generatedfrom those signals coming from the area of about 480 vertically arrangedpixels. When such a camera is used to photograph a still image, it isrelatively easy to generate a static image signal conforming to, forexample, JPEG (Joint Photographic Expert Group) from the signals comingfrom the same pixel area that is used to take a moving image. A problemremains, however, that the number of vertically arranged pixels obtainedis limited to around 480, making it impossible to produce more detailedstatic image signals.

[0005] In a camera having an image sensing device with the area ofpixels for image stabilization mentioned above, a method of alleviatingthis problem may involve using the entire area of effective pixelsincluding the area of image stabilization pixels in photographing astill image. Even when photographing a still image, however, thephotographed image needs to be monitored for check and, for thatpurpose, it is necessary to generate signals conforming to thetelevision system from signals read out from all effective pixels.

[0006] An example of such a conventional camera has been proposed inJP-A-11-187306. In the camera disclosed in this publication, signalsfrom all the effective pixels are read out taking two or more times thefield period of the television system, stored in a memory means such asa field memory, and then subjected to interpolation processing fortransformation into signals conforming to the field cycle and horizontalscan cycle of television.

[0007] This conventional camera, however, requires a large processingcircuit, such as field memory, for signal conversion. Another drawbackis that the image sensing device readout cycle is a plurality of timesthe field cycle, degrading the dynamic resolution. Even with the use ofthis circuit, the number of pixels obtained as the static image signalsis limited to the number of effective pixels used for moving videos plusthe area of image stabilization pixels.

[0008] In a digital still camera designed for taking still images, therehas been a trend in recent years toward an increasing number of pixelsused on the moving video image sensing device in order to obtain higherresolution static image signals. When taking a moving image ormonitoring the video, it is necessary to generate signals that conformto the television system. The number of pixels on such an image sensingdevice, however, does not necessarily match the number of scanning linesof the television system and therefore some form of conversion means isrequired.

[0009] The conversion means may involve, as in the video camera with thearea of image stabilization pixels, reading out signals from the imagesensing device taking a longer time than the field cycle andinterpolating them to generate television signals. This method has, inaddition to the problem described above, a drawback that the readoutcycle increases as the number of pixels increases, further degrading thedynamic resolution.

[0010] To mitigate this problem, JP-A-9-270959 discloses an apparatuswhich mixes together or culls the pixel signals inside the image sensingdevice to reduce the number of signals to be read and therefore the readcycle. Although this apparatus alleviates the problem of the degradeddynamic resolution, it requires a large processing circuit such as fieldmemory to perform time-axis transformation to generate signalsconforming to the television system and the image sensing device itselfneeds to have a special structure for performing desired mixing andculling.

SUMMARY OF THE INVENTION

[0011] The present invention relates to a photography of video cameras,camcorders, digital still cameras and others using a solid-state imagesensing device, and more particularly to an electric camera using asolid-state image sensing device with a large number of pixels.

[0012] The conventional electric cameras, as described above, havedrawbacks that when taking a still picture by using a video camera, thenumber of pixels is not sufficient and that when taking a moving imagewith a still camera, the associated circuit inevitably increases and thedynamic image quality deteriorates. Taking both moving and static imagesof satisfactory quality with a single camera is difficult to achieve. Inaddition to solving the above problems, to obtain good dynamic picturequality by using an image sensing device having a large number of pixelsintended for still images requires extracting a pixel area that is usedto realize an image stabilizing function. The conventional art andcameras do not offer a means to accomplish this function.

[0013] An object of the present invention is to provide an electriccamera that solves these problems and which uses an image sensing devicewith a sufficient number of pixels for still images and enables takingof highly detailed still images and a moving video taking with reducedimage quality degradation without increasing circuitry such as fieldmemory. It is also an object of the present invention to provide anelectric camera that can also realize the image stabilizing function.

[0014] According to one aspect of this invention, the electric camera torealize the above objectives has: an image sensing device with a lightreceiving surface having N vertically arranged pixels and an arbitrarynumber of pixels arranged horizontally, N being equal to or more thanthree times the number of effective scanning lines M of a display screenof a television system; a driver to drive the image sensing device tovertically mix or cull signal charges accumulated in individual pixelsof every K pixels to produce a number of lines of output signals whichcorresponds to the number of effective scanning lines M, K being atleast one of integers equal to or less than an integral part of aquotient of N divided by M; and a signal processing unit to generateimage signals by using the output signals of the image sensing device.

[0015] As explained above, since this invention eliminates the limit onthe number of vertically arranged pixels, an electric camera can beprovided which enables taking of highly detailed still images and asatisfactory moving video taking by using an image sensing device with alarge enough pixel number even for still images.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016]FIG. 1 is a block diagram showing the configuration of a firstembodiment of an electric camera according to the present invention.

[0017]FIG. 2 is a schematic diagram showing the structure of an imagesensing device in the first embodiment of the electric camera of theinvention.

[0018]FIG. 3 is a drive pulse timing diagram in the first embodiment ofthe electric camera of the invention.

[0019]FIG. 4 is a schematic diagram showing a mixing operation in thefirst embodiment of the electric camera of the invention.

[0020]FIG. 5 is a schematic diagram showing a readout area in the firstembodiment of the electric camera of the invention.

[0021]FIG. 6 is a schematic diagram showing a mixing operation in thefirst embodiment of the electric camera of the invention.

[0022]FIG. 7 is a block diagram showing the configuration of a secondembodiment of an electric camera according to the present invention.

[0023]FIG. 8 is a schematic diagram showing a mixing operation in thesecond embodiment of the electric camera of the invention.

[0024]FIG. 9 is a schematic diagram showing a readout area in the secondembodiment of the electric camera of the invention.

[0025]FIG. 10 is a schematic diagram showing the structure of an imagesensing device in a third embodiment of the electric camera according tothe present invention.

[0026]FIG. 11 is a drive pulse timing diagram in the third embodiment ofthe electric camera of the invention.

[0027]FIG. 12 is a schematic diagram showing an interpolation operationin the third embodiment of the electric camera of the invention.

[0028]FIGS. 13A and 13B are schematic diagrams showing the arrangementof color filters in the image sensing device in a fourth embodiment ofthe electric camera according to the present invention.

DESCRIPTION OF THE EMBODIMENTS

[0029] Now embodiments of the present invention will be described byreferring to the accompanying drawings. FIG. 1 is a block diagramshowing the configuration of one embodiment of an electric cameraaccording to the invention.

[0030] In FIG. 1, reference number 1 represents a lens, 2 an aperture, 3an image sensing device, 4 a drive circuit, 5 a gain adjust circuit, 6an analog-digital (A/D) conversion circuit, 7 a signal processingcircuit, 8 a vertical interpolation circuit to perform interpolation ina vertical direction, 9 a horizontal interpolation circuit to performinterpolation in a horizontal direction, 10 a recording unit includingrecording media such as magnetic tape, semiconductor memory and opticaldisk to record a video signal, 11 a control circuit to control theseconstitutional elements according to the operating state, 12 an encodercircuit to modulate the video signal into a standard television signal,13 a digital-analog (D/A) conversion circuit, 14 a mode selector switchto change over the operation mode between the moving video taking andthe still image taking, 15 a record button to start or stop therecording, 16 a and 16 b gyro sensors to detect verticalimage-unstability and lateral image-unstability, respectively, and 17 animage-unstability decision circuit to determine the image-instabilityfrom signals output from the gyro sensors.

[0031] In the above configuration, light coming from the lens 1 throughthe aperture 2 is focused on a light receiving surface of the imagesensing device 3 where it is converted into an electric signal. In thisembodiment the image sensing device 3 is of a CCD type. FIG. 2 shows thestructure of this image sensing device 3. In FIG. 2, denoted 30 arepixels each formed of a photodiode, which are arranged horizontally andvertically in a grid pattern. On these grid-arrayed pixels three typesof color filters that pass yellow (Ye), green (G) and cyan (Cy),respectively, are arranged in such a way that the combination of thesethree colors is repeated horizontally every three pixels and that thefilters of the same colors are lined vertically in so-called verticalstripes. Although an arbitrary number of pixels may be used, thisembodiment has an array of 1200 pixels vertically and 1600 pixelshorizontally. A vertical transfer unit 32 is a CCD which is driven bythree phase pulses V1, V2, V3. This CCD has a three-gate structure inwhich each pixel corresponds to three phase pulses and thus canvertically transfer a signal charge of each pixel independently.Transfer gates 31 for transferring the charge of each pixel to thevertical transfer unit 32 are commonly connected to a gate of thevertical transfer unit 32 that corresponds to the V2 pulse. An operationto transfer the charge from each pixel to the vertical transfer unit 32in response to a peak value of the pulse applied to the commonlyconnected gate and an operation to transfer the charge vertically areperformed separately. A horizontal transfer unit 33 horizontallytransfers the charges supplied from the vertical transfer units 32 andoutputs them successively through an output amplifier 34 from the outputterminal.

[0032] Referring back to FIG. 1, the operation performed when the movingvideo mode is selected by the mode selector switch 14 will be explained.The number of vertically arranged pixels on the image sensing device inthis embodiment is 1200, so if the number of effective scanning lines inthe field of the NTSC system is assumed to be 240 lines, then verticallymixing five pixels (=1200 pixel rows/240 scanning lines) can match thenumber of lines of output signals from the image sensing device to thenumber of effective scanning lines.

[0033] However, in this embodiment, to realize the image stabilizingfunction described later, four vertically arranged pixels are mixedtogether during motion image taking mode. When four vertically arrangedpixels are to be cyclically mixed together, the signals from the ar a of960 pixels (=240 scanning lines×4 pix ls) out of the 1200 verticallyarranged pixels are used as effective signals and the remaining 240pixels (=1200 (all pixels) 960 (effective pixels)) are not used forimage forming. FIG. 3 shows the timing of a vertical drive pulse for theimage sensing device in this operation mode, with V1, V2 and V3representing three phase drive pulses applied to each gate of the CCD orvertical transfer unit 32.

[0034] In FIG. 3, in a period T1 included in the vertical blankingperiod, the drive pulse V2 is held high to transfer the signal chargeaccumulated in each pixel to under the V2 gate of the vertical CCD.Next, in a period T2, while the V2 pulse is still at middle level, theV3 pulse is raised from low level to middle level; next, while the V3pulse is at middle level, the V2 pulse is changed from middle level tolow level, after which the V1 pulse is changed from low level to middlelevel; next, while the V1 pulse is at middle level, the V3 pulse ischanged from middle level to low level, after which the V2 pulse ischanged to middle level and finally the V1 pulse is changed from middlelevel to low level. With this sequence of pulse operations, the signalcharges under the V2 gate for one pixel row are transferred and heldagain under the V2 gate.

[0035] By repeating this series of operations, the signal charges for adesired number of pixel rows can be transferred. In FIG. 3, during aperiod T3 included in the vertical blanking period before the verticaleffective scanning period (the vertical scanning period minus thevertical blanking period which corresponds to the actually displayedimage) and during a period T4 included in the vertical blanking periodafter the vertical effective scanning period, the above transferoperation for one pixel row is repeated a total of 240 times to transferthe signal charges of the 240 pixel rows not used for image generationto the horizontal transfer unit 33 during the vertical blanking period.For example, if this transfer operation is performed 120 times duringthe period T3 and 120 times during the period T4, the signal chargesfrom upper 120 pixel rows and lower 120 pixel rows on the lightreceiving surface are transferred to the horizontal transfer unit 33during the period T3 and period T4 within the vertical blanking period.During each of the subsequent periods T5 and T6 in the vertical blankingperiod, the horizontal transfer unit 33 is driven for a predeterminedperiod to output the charges transferred to the horizontal transfer unit33 from the output terminal. These charges are not used as valid signalsas they are output during the vertical blanking period.

[0036] Next, in the vertical effective scanning period of FIG. 3, theabove one-pixel-row transfer operation is performed four times duringeach horizontal blanking period to transfer the signal charges of fourpixel rows to the horizontal transfer unit 33 where they are mixedtogether. Then, during a horizontal effective scanning period (thehorizontal scanning period minus the horizontal blanking period whichcorresponds to the actually displayed image), the horizontal transferunit 33 is driven to read out the signal charges from the horizontaltransfer unit to produce an output signal conforming to the televisionsystem. If the above operation is performed on the A field and if, onthe B field, the number of pixel rows transferred during the period T3is set to 122 rows and that during the period T4 is set to 118 rows,then the combination of four pixel rows to be cyclically mixed togethershifts by two rows between the two fields, thus allowing the interlacedscanning to be performed as shown in FIG. 4. (FIG. 4 shows the lightreceiving surface of the image sensing device and its relation to thedisplayed screen is vertically inverted.)

[0037] Let us return to FIG. 1. The output signal from the image sensingdevice 3 is adjusted in gain by the gain adjust circuit 5 and thenconverted by the A/D conversion circuit 6 into a digital signal. Thedigital signal is then processed by the signal processing circuit 7 thatperforms color signal processing and luminance signal processing, suchas generation of color signals, gamma correction, white balanceprocessing and outline enhancement. The image sensing device in thisembodiment has an array of vertical stripes of yellow (Ye), green (G)and cyan (Cy) color filters, so the color signals for Ye, G and Cy areobtained as a series of color points from one line of output signals atall times no matter how many pixels are vertically combined. From thesecolor signals three primary color signals R, G, B can be obtained fromthe following calculations.

[0038] R=Ye−G

[0039] B=Cy−G

[0040] G=G

[0041] The R, G and B signals undergoes the white balance processing andgamma correction processing in the signal processing circuit 7 and arethen converted into color difference signals such as R-Y, B-Y or U andV. The luminance signals and the color difference signals are thenentered through the vertical interpolation circuit 8 into the horizontalinterpolation circuit 9. In this operation state the signals just passthrough the vertical interpolation circuit 8 without being processed.The horizontal interpolation circuit 9 performs interpolation on thesignals in the horizontal direction.

[0042]FIG. 5 shows the light receiving surface of the image sensingdevice. As described above, in the operating state of this embodiment,the signals read out during the vertical effective scanning periodcorrespond to an area having 960 of the 1200 vertically arranged pixelsand a horizontal width of 1600 pixels, as shown shaded at A in FIG. 5.If the entire light receiving surface of the image sensing device has a4-to-3 (width to height) aspect ratio, the shaded area A is morelaterally elongate than this aspect ratio. Hence, if the signals of allhorizontal pixels of the light receiving surface are displayed, forexample, on an NTSC standard television monitor with the 4-to-3 aspectratio, the image displayed is compressed horizontally and looksvertically elongate, compared with the original image. It is thereforenecessary to output during the horizontal effective scanning period onlythose signals coming from a pixel area with the horizontal widthconforming to the aspect ratio of the television system, as shown by ashaded area B. When the television system has an 4-to-3 aspect ratio,the number of pixels in the horizontal width of the shaded area B is1280 (=960 (vertical effective pixels)×4/3).

[0043] Returning back to FIG. 1, the horizontal interpolation circuit 9performs interpolation processing on the signals from the horizontal1280 pixels to expand the signals so that they can be output over theentire horizontal effective scanning period. It also performs switchingamong different clocks as required. With the above operation, an areahaving 960 pixels in height and 1280 pixels in width is demarcated fromthe light receiving surface as signals conforming to the televisionsystem. Then, the luminance signal and the color difference signal areencoded by the encoder circuit 12 into television signals, which arethen converted by the D/A conversion circuit 13 into analog signals foroutput. When the recording is specified by the record button 15, thesignals are recorded by the recording unit 10. At this time, the signalsmay be compressed in the MPEG (Moving Picture Expert Group) format.

[0044] Next, the image stabilizing operation will be explained.Image-unstability information obtained by the gyro sensors 16 a, 16 bthat detect vertical and horizontal image-unstabilities is entered intothe image-unstability decision circuit 17, which checks the receivedinformation for the amount and direction of the image-unstability andconverts them into the number of pixels in vertical and horizontaldirections on the light receiving surface of the image sensing device.Based on the converted pixel numbers, the position of an extracted area(effective pixel area) on the light receiving surface is shifted in adirection that cancels the image-unstability. This can correct theimage-unstability. The positional shifting of the extracted area isperformed as follows. The shifting in the vertical direction can be madeby changing the number of pixel rows transferred during the periods T3and T4 in FIG. 3 and the shifting in the horizontal direction made bychanging the interpolation start position in the horizontalinterpolation circuit 9.

[0045] The operation during the moving video mode has been describedabove. Next, the operation performed when the static image mode isselected by the mode selector switch 14 will be explained.

[0046] In the static image mode, too, until the recording is requestedby the record button 15, the camera outputs signals compatible with thetelevision system to monitor the angle of view. Unlike the moving videophotographing, all of the effective pixels on the image sensing deviceare used in this embodiment during the still image photographing toproduce signals with as high a resolution as possible. Hence, during themonitoring the television signals need to be generated from the signalscoming from the entire pixel area.

[0047] The image sensing device of this embodiment has 1200 verticallyarranged pixels, and the number of lines of output signals from theimage sensing device can be made to match the number of effectivescanning lines of NTSC system, which is assumed to have 240 scanninglines, by vertically mixing five pixels (=1200/240). To make the imagesensing device operate in this manner, the one-pixel-row transferoperation is performed five times during each horizontal blanking periodin the vertical effective scanning period shown in the pulse timingdiagram of FIG. 3. With this operation, the signal charges of five pixelrows can be mixed by the horizontal transfer unit 33. As for thetransfer operations during the periods T3 and T4 in the verticalblanking period, because the interlaced scanning is carried out, onlytwo pixel rows are transferred during the period T3 on the B field, withno transfer operations performed in other vertical blanking periods (Inthis embodiment, 1200/240=5 with no remainder produced, so no furthertransfer is necessary; if, however, a remainder occurs, the remainingpixels need only be transferred during the periods T3 and T4).

[0048] The charges mixed by the horizontal transfer unit 33 are read outby driving the horizontal transfer unit 33 during the horizontaleffective scanning period. With the above operations, the signal chargesof all pixels on the image sensing device can be read out in a mannerconforming to the television system. The output signal from the imagesensing device 3 is, as during the moving image photographing, adjustedin gain by the gain adjust circuit 5 and converted by the A/D conversioncircuit 6 into a digital signal, which is then subjected to the colorsignal processing and the luminance signal processing in the signalprocessing circuit 7 before being entered into the verticalinterpolation circuit 8. During the static image monitoring, thevertical interpolation circuit 8 performs a vertical gravity centercorrection on the received signals.

[0049]FIG. 6 shows combinations of pixels to be cyclically mixed on theA field and the B field and also the vertical position of the gravitycenter of the mixed signals. In the interlaced scanning, scanning linesof the A field and the B field are located at the centers of adjoiningscanning lines on other field. Hence, the signal samplings in the camerasystem for the two fields must be 180 degrees out of phase in thevertical direction. In the operating state of this embodiment, however,because five pixels are mixed together, the gravity centers of theoutput signals for the A field and the B field are deviated 36 degrees(=½ pixel or {fraction (1/10)} the line-to-line distance on the samefield) from the ideal sampling phase difference of 180 degrees, as shownin FIG. 6. To correct this requires generating a signal from twoadjoining output lines by interpolation. For example, if we let an nthoutput line on a field be Sn and an (n+1)th output line on the samefield be Sn+1, then a signal Sn′ obtained by calculatingSn′=(Sn×{fraction (9/10)})+(Sn+1×{fraction (1/10)}) is one whose gravitycenter is shifted by {fraction (1/10)} output line from the gravitycenter of the nth output line toward the (n+1)th output line. The abovecalculation can also be performed on the signals for the B field tocorrect the gravity center deviation of sampling. In this embodiment,however, to equalize the effects of interpolation of the A field and theB field, the following calculations are performed to correct the nthoutput line by {fraction (1/20)} line toward the (n−1) line on the Afield and by {fraction (1/20)} line toward the (n+1)th line on the Bfield.

[0050] A field: Sn′=(Sn×{fraction (19/20)})+(Sn−1×{fraction (1/20)})

[0051] B field: Sn′=(Sn×{fraction (19/20)})+(Sn+1×{fraction (1/20)})

[0052] While this embodiment performs the interpolation based on thecalculation of two adjoining lines of output signals, a greater numberof lines may be used for the interpolation processing. The output signalof the vertical interpolation circuit 8 is supplied to the horizontalinterpolation circuit 9, which in this operating state does nothing butpasses the signal. Then, as in the case of the moving imagephotographing, the signal is encoded by the encoder circuit 12 into atelevision signal, which is then converted by the D/A conversion circuit13 into an analog signal for output. As described above, the televisionsignals can be generated from all of the pixel area of the image sensingdevice also during the static image mode.

[0053] Next, the operation performed when the recording is requested bythe record button 15 will be described. During the monitoring in thestatic mode, the signals are mixed together inside the image sensingdevice to reduce the number of signals and thereby generate televisionsignals. During recording, however, the mixing processing is notperformed and all the pixel signals need to be read out independently ofeach other in order to produce high resolution signals. To realize this,the one-pixel-row transfer operation is performed only once during eachhorizontal blanking period in the vertical effective scanning periodshown in the pulse timing diagram of FIG. 3. This causes only the signalcharges for one pixel row to be transferred into the horizontal transferunit 33, which is then driven to read out the signal charges for onepixel row. Repeating this operation the number of times equal to thenumber of vertically arranged pixel rows enables the signal charges ofall pixel rows to be read out independently of each other. The transferoperation is not done during the periods T3 and T4 in the verticalblanking period.

[0054] The signal charges thus read out are adjusted in gain by the gainadjust circuit 5 and converted by the A/D conversion circuit 6 intodigital signals, which are then subjected to the color signal processingand the luminance signal processing in the signal processing circuit 7,after which the signals are supplied through the vertical interpolationcircuit 8 and the horizontal interpolation circuit 9 to the recordingunit 10 where they are recorded. At this time, no interpolationprocessing is performed by the vertical interpolation circuit 8 orhorizontal interpolation circuit 9. The recording unit 10 may compressthe signals in the JPEG (Joint Photographic Expert Group) format, forexample. Because during the static image recording, no television signalcan be generated, an image immediately before starting the recording ora single color image is output as the television signal for monitoringpurpose. With the above operation, high resolution signals obtained fromall the pixels of the image sensing device can be recorded. Although inthis embodiment the recording unit is used commonly for the moving videomode and for the static image mode, dedicated recording units may beprovided separately for these modes.

[0055] As explained above, since there is no limit on the number ofvertically arranged pixels in this embodiment, an image sensing devicewith a large enough pixel number even for still images can be used tophotograph highly detailed still images and satisfactory moving images.

[0056] Further, the signal mixing and the vertical signal transferduring the vertical blanking period allow the signals from the imagesensing device with a large number of pixels to be read out in a mannerthat conforms to the television system. This in turn can reduce imagequality degradation and realize the moving image photographing with animage stabilizing function and the monitoring during still imagephotographing.

[0057] Only the output signals from that horizontal segment whichvirtually corresponds to the television system's aspect ratio withrespect to the vertical segment are extracted and output over the entirehorizontal effective scanning period of the television system. Thisensures that the output signals obtained conform to the televisionsystem's aspect ratio regardless of the extracted vertical segmentposition.

[0058] Further, the image sensing device is driven in such a way as toshift the position of the pixels to be cyclically mixed together everydisplay cycle of the television system in order to output interlacedsignals with this arrangement, the interlaced scanning can be performedeven when an image sensing device with a large number of pixels is used.

[0059] Further, the output signals produced by the mixing areinterpolated so that the gravity centers of the output signalsinterlaced every display cycle have a phase difference of 180 degrees inthe vertical direction. This ensures that the interlaced output signalshave no deviation from the ideal 180-degree phase difference duringinterlacing even when an interlace phase deviation would normally occur,as when odd numbered pixels are mixed together.

[0060] In this embodiment, the image sensing device has 1200 verticallyarranged pixels, and four pixels are mixed together during the movingvideo mode and five pixels during the static image mode. Because thearea of image-stabilization pixels may or may not be used and set to anydesire size, the number of pixels to be cyclically mixed together ineach mode needs only to be equal to or less than the integral part of aquotient of the number of vertically arranged pixels divided by thenumber of television system's effective scanning lines (in the aboveexample, 5 or less). (The number of vertically arranged pixels does notneed to be divisible and, in the above example, may be more than 1200).

[0061] The number of vertically arranged pixels for static imagephotographing needs only to be three or more times the number ofeffective scanning lines on each field of the television system. In thisembodiment the vertically adjoining pixels are mixed together to reducethe number of output lines from the image sensing device during thevertical effective scanning period. The number of lines of outputsignals can also be reduced by a so-called culling operation, by whichonly one line of signal charges of pixels is read out for everypredetermined number of lines.

[0062] While in this embodiment the vertical transfer unit of the imagesensing device is formed as a CCD that is driven by three phase pulsesfor each pixel, the image sensing device may have any desired structureas long as it can realize the mixing or culling of pixels that meets theabove conditions.

[0063] Although this embodiment described the case of NTSC system, theinvention can also be applied to other television systems, such as PALstandard, with different numbers of effective scanning lines.

[0064] In summary, a variety of constructions essentially equal in theworking principle to this embodiment can be realized by the use of animage sensing device that has an arbitrary number of vertically arrangedpixels N three or more times the number of effective scanning lines M ofeach field of the television system and which allows the vertical mixingor culling of that number of pixels which is at least one of integersequal to or less than the integral part of a quotient of N divided by M.

[0065] Next, another embodiment of the present invention will bedescribed by referring to FIG. 7 showing the configuration of theembodiment. The configuration shown in FIG. 7 differs from that of FIG.1 in that it has a view angle change switch 18. In FIG. 7 constitutionalelements identical with those shown in FIG. 1 are assigned likereference numbers and explanations on the constitutional elementsperforming the same operations as those in FIG. 1 are omitted here.

[0066] The operations of the moving video mode, the monitoring duringthe static image mode and the static image recording are similar innormal condition to the operations of the previous embodiment which wasexplained with reference to the configuration diagram of FIG. 1. Anoperation performed when during the moving video mode a request tochange the angle of view is made by the view angle change switch 18 willbe described.

[0067] In the normal condition of this embodiment, as explained in theprevious embodiment, the mixing of four vertically arranged pixels, thevertical transfer during the vertical blanking period and the horizontalinterpolation processing are performed to extract an area of 960 pixelsin height and 1280 pixels in width from the entire pixel area togenerate television signals. When a view angle change (which means azooming function without image quality degradations in the verticaldirection) is requested by the view angle change switch 18, threevertically arranged pixels are mixed together and the signals from theexcess vertically arranged pixels are read out during the verticalblanking periods before and after the vertical effective scanningperiod.

[0068] In this embodiment, signals of 480 pixels (=1200−240×3) or 160lines of output signals after mixing (=480/3) need to be read out duringthe vertical blanking period. This allows the signals of 720 verticallyarranged pixels to be read out as 240 lines of output signals conformingto the television system. To carry out this reading requires, in thepulse timing diagram of FIG. 3, transferring the signals of three pixelrows during each horizontal blanking period in the vertical effectivescanning period and also transferring a total of 480 pixel rows (=160lines of output signals) during the T3 and T4 periods in the verticalblanking period. The combinations of pixels to be cyclically mixedtogether are changed from one field to another to achieve the interlacedscanning.

[0069] The output signals of the image sensing device 3 are supplied tothe gain adjust circuit 5. Because the signal level produced as a resultof the 3-pixel mixing is 3/4 the signal level of the 4-pixel mixing, theagain of the gain adjust circuit 5 is increased to 4/3 the gain of the4-pixel mixing to make the 3- and 4-pixel-mixed input signal levels tothe subsequent circuit equal. Then, the signals are processed by the A/Dconversion circuit 6 and the signal processing circuit 7 before beingsupplied to the vertical interpolation circuit 8. The combinations ofpixels to be cyclically mixed on the A field and the B field and thevertical positions of the gravity centers of the mixed signals are shownin FIG. 8. As in the static image monitoring of the previous embodiment,the phase difference between the two fields is 180 degrees. Because thesampling phases of the fields are deviated from the ideal 180-degreephase difference, the vertical position of the gravity centers arecorrected by the vertical interpolation circuit 8. The amount of phasedeviation in this operating state is 60 degrees (=½ pixel or ⅙ theline-to-line distance on the same field). To correct the phase deviationevenly on the both fields, the following calculations should beperformed.

[0070] A field: Sn′=(Sn×{fraction (11/12)})+(Sn−1×{fraction (1/12)})

[0071] B field: Sn′=(Sn×{fraction (11/12)})+(Sn+1×{fraction (1/12)})

[0072] As described earlier, the interpolation processing may use threeor more output lines. Next, the horizontal interpolation circuit 9horizontally expands the signals from that horizontal segment whichcorresponds to the 4-to-3 aspect ratio with respect to the 720vertically arranged pixels (i.e., signals from a horizontal 960-pixelsegment (=1600×720/1200) in this operating state) so that the expandedsignals can be output during the entire horizontal effective scanningperiod. With the above operation, an area of 720 pixels in height and960 pixels in width can be extracted from the light receiving surface.

[0073] Next, when the view angle change is requested again by the viewangle change switch 18, two vertically arranged pixels are mixedtogether, an area of 480 vertically arranged pixels is read out duringthe vertical effective scanning period, and the horizontal interpolationcircuit 9 expands the signals from the horizontal 640-pixel segment andoutputs the expanded signals during the entire horizontal effectivescanning period to extract an area of 480 pixels in height and 640pixels in width. (During the two-pixel mixing, because the interlacephase deviation does not occur, the gravity center position correctionby the vertical interpolation circuit 8 is not performed.) If a furtherview angle change is requested by the view angle change switch 18, theoperation is restored to a normal state where four vertically arrangedpixels are mixed together.

[0074] As a result of the above operation, the area extracted from thelight receiving surface of the image sensing device can be changed tothree different areas: (A) 960 pixels high by 1280 pixels wide, (B) 720pixels high by 960 pixels wide and (C) 4880 pixels high by 640 pixelswide. That is, the angle of view can be changed to three differentangles. With the area A produced by the 4-pixel mixing taken as areference, the area B produced by the 3-pixel mixing can provide animage enlarged by 1.33 times and the area C produced by the 2-pixelmixing can provide an image enlarged by two times. It should be notedhere that because the three different areas are chosen by changing thenumber of pixel to be cyclically mixed together in order to make thenumber of the effective output lines from the imaging device agree withthe number of the effective scanning lines of the television system, theangle of view can be changed while maintaining a good image with noimage quality degradation in the vertical direction, when compared withan ordinary so-called digital zoom which generates effective scanninglines of signals by interpolating a small number of output lines. Duringthe monitoring of a static image, too, it is possible to perform thesimilar operation of changing the angle of view by changing the numberof pixel rows to be cyclically mixed together.

[0075] As described above, in addition to the advantages provided by theprevious embodiment, this embodiment can also realize the view anglechange with little image quality degradation even for still images byusing an image sensing device with a large number of pixels and changingthe number of pixels to be cyclically mixed together.

[0076] Further, because changes in signal level caused when the numberof pixels to be cyclically mixed is changed are absorbed by the gainadjust means, the input signal level to the subsequent signal processingmeans can be kept constant.

[0077] While in this embodiment, the view angle change is performed bythe view angle change switch 18, the angle of view may be changedcontinuously by a zoom switch. In this case, when the magnificationfactor does not reach the value that is obtained by changing the pixelmixing, the digital zoom performs the ordinary interpolation processing.In this embodiment, when the magnification factor is 1 or more and lessthan 1.33, the 4-pixel mixing is performed; for the factor of 1.33 ormore and less than 2, the 3-pixel mixing is done; and for the factor of2 or higher, the 2-pixel mixing is carried out. The mixing operation maybe interlocked with an optical zooming mechanism.

[0078] Regardless of the number of pixels in the image sensing device,the structure of the image sensing device or the television systememployed, this embodiment, as in the previous embodiment, may also usean image sensing device that has an arbitrary number of verticallyarranged pixels N three or more times the number of effective scanninglines M of each field and which allows the vertical mixing or culling ofthose numbers of pixels which are at least two of integers equal to orless than the integral part of a quotient of N divided by M. The use ofthis image sensing device can form a variety of constructionsessentially equal in the working principle to this embodiment.

[0079] Next, a further embodiment of the present invention will bedescribed. The overall configuration of this embodiment is similar tothat of FIG. 1, except that the inner structure of the image sensingdevice 3 is different. The configuration of the image sensing device inthis embodiment is shown in FIG. 10. In FIG. 10, denoted 30 are pixelsformed of photodiodes, which are arranged horizontally and vertically ina grid pattern. On these grid-arrayed pixels three types of colorfilters that pass yellow (Ye), green (G) and cyan (Cy), respectively,are arranged in so-called vertical stripes.

[0080] In this embodiment, the image sensing device has an array ofpixels measuring 864 pixels vertically and 1152 pixels horizontally. Avertical transfer unit 32 is a CCD which is driven by six phase pulsesV1, V2, V3, V4, V5, V6. This CCD has a two-gate structure in which eachpixel corresponds to two phase pulses and six gates corresponding to thesix phase pulses are repeated for every three pixels. Transfer gates 31for transferring the signal charge of each pixel to the verticaltransfer unit 32 are commonly connected to respective gates of thevertical transfer unit 32 corresponding to the pulses V1, V3, V5. Anoperation to transfer the signal charge from each pixel to the verticaltransfer unit 32 in response to peak values of pulses applied to thecommonly connected gates and an operation to transfer the chargevertically are performed separately.

[0081] A horizontal transfer unit 33 horizontally transfers the chargessupplied from the vertical transfer units 32 and outputs themsuccessively through an output amplifier 34 from the output terminal.This image sensing device, unlike the one in the previous embodiment,cannot vertically transfer all pixels independently of each other, butcan mix together the signal charges of three vertically adjoining pixelsinside the vertical transfer unit 32 before transferring them.

[0082] First, the operation performed in this embodiment during themoving video mode will be explained. In the image sensing device of thisembodiment the effective number of vertically arranged pixels is 864. Ifthree pixels are vertically mixed, the signals of the 720 (=240×3) ofthe 864 vertically arranged pixels can be used as the effective signalsand the remaining 144 (=864−720) pixels can be used as theimage-unstability correction pixel area.

[0083]FIG. 11 shows the timings of vertical drive pulses for the imagesensing device of FIG. 10 during this operation mode, with V1, V2, V3,V4, V5, V6 representing the six phase drive pulses applied to therespective gates of the CCD or vertical transfer unit 32. In FIG. 11,during a period T1 included in the vertical blanking period, the drivepulses V1, V3 and V5 are held high to cause the signal charge of eachpixel to be transferred to under the V1, V3 and V5 gates of the verticalCCD. Then, the V2 and V4 pulses are changed from low level to middlelevel to mix the charges of the adjoining three pixels. After themixing, the V5 pulse is changed from middle level to low level to holdthe mixed signal charges under the V1, V2, V3, V4 gates.

[0084] Next, a series of operations performed during a period T2(changing the drive pulses from middle level to low level or from lowlevel to middle level in the order of V1, V2, V3, V4, V5 and V6) causesone mixed output line (3 pixel rows) to be transferred and held againunder the V1, V2, V3, V4 gate. By repeating this series of operations, adesired number of output lines of mixed signal charges can betransferred.

[0085] In FIG. 11, during a period T3 included in the vertical blankingperiod before the vertical effective scanning period and during a periodT4 included in the vertical blanking period after the vertical effectivescanning period, the transfer operation for one output line is repeateda total of 144 times to transfer 144 output lines of signal charges notused for image forming to the horizontal transfer unit 33 at high speedduring the vertical blanking periods. During subsequent periods T5 andT6 in the vertical blanking periods, the horizontal transfer unit 33 isdriven for predetermined periods to output the signal chargestransferred to the horizontal transfer unit 33 from the output terminal.

[0086] Next, in the vertical effective scanning period of FIG. 11, theone-output-signal-line transfer operation is performed during eachhorizontal blanking period. Then, during the horizontal effectivescanning period, the horizontal transfer unit 33 is driven to read outthe signal charges from the horizontal transfer unit 33. With thisoperation the signal charges of three pixels mixed together can be readout in a way conforming to the television system. As shown in FIG. 11,the signals for the A field are mixed by changing the V2 and V4 pulsesto middle level after transferring the signals from the pixels to thevertical transfer unit 32. The signals for the B field, on the otherhand, are mixed by changing the V2 and. V6 pulses to middle level. Withthis mixing method, the combinations of pixels to be cyclically mixedtogether can be changed from one field to another, thereby realizing theinterlaced scanning. The output signals from the image sensing deviceare processed in the similar manner to that of the previous embodiment.A vertical interpolation circuit 8 performs the gravity centercorrection, as in the 3-pixel mixing in the previous embodiment, and ahorizontal interpolation circuit 9 performs interpolation processing tomatch the aspect ratio with that of the television system.

[0087] Next, the operation during the monitoring in the static imagemode will be explained. It is assumed that the still image photographingis done by using all effective pixels of the image sensing device, as inthe previous embodiment. The image sensing device of this embodiment has864 vertically arranged pixels and, when 3-pixel mixing is done as inthe moving video taking, the number of output lines is 288 (=864/3),which means that these signal lines cannot be read out in a mannerconforming to the television system. Hence, during the monitoring in thestatic image mode, vertical 6-pixel mixing is performed. The 6-pixelmixing can be achieved by transferring to the horizontal transfer unit33 in each horizontal blanking period two output lines of signal chargeseach of which line has been generated by vertically mixing three pixelswithin the vertical transfer unit 32. The 6-pixel mixing can reduce thenumber of output lines from the image sensing device down to 144(=864/6) lines. The output signals of the image sensing device that werereduced to 144 output lines are interpolated by the verticalinterpolation circuit 8 to transform the 144 output lines of signalsinto 240 lines of signals, which conform to the television system. Togenerate 240 lines of signals from the 144 lines requires interpolationprocessing that generates five lines from three lines (144/240=3/5).

[0088]FIG. 12 shows how the interpolation is performed using twoadjoining output lines. Let three output lines of the image sensingdevice be n, n+1 and n+2. The five output lines of signals can begenerated from the following calculations.

[0089] n′=n

[0090] n′+1=n/2+(n+1)/2

[0091] n′+2=n+1

[0092] n′+3=(n+1)/2+(n+2)/2

[0093] n′+4=n+2

[0094] Three or more output lines of signals may be used forinterpolation processing. With the above operation, television signalscan be generated by using signals of all pixels of the image sensingdevice also during the monitoring in the static image mode.

[0095] Next, the operation performed when the recording is requested bythe record button 15 will be explained. In the recording process, themixing processing is not performed and signals of all pixels need to beread independently of each other in order to obtain high-resolutionsignals. To realize this, the aperture 2 is first closed and then,during the period T2 in the pulse timing diagram of FIG. 11, only the V1pulse is raised to high level to transfer the signal charge of only thepixel adjacent to the V1 gate to the vertical transfer unit 32. Then,the vertical transfer unit 32 and the horizontal transfer unit 33 aresuccessively driven to read out the signal charges. Similarly, the V3pulse is raised to high level to read the signal charge of the pixeladjacent to the V3 gate, followed by raising the V5 pulse to high levelto read the signal charge of the pixel adjacent to the V5 gate. With theabove processing, the signal charges of all pixels can be read outindependently in three successive operations. The signal charges thusread out are recorded in the recording unit 10. At this time, they arerearranged properly to reconstruct the pixel arrangement on the lightreceiving surface of the image sensing device.

[0096] As described above, this embodiment offers the followingadvantages. If the number of vertically arranged pixels is not anintegral multiple of the number of scanning lines of the televisionsystem, the signals conforming to the television system can be generatedfrom the whole area of effective pixels by performing the pixel mixingand the vertical interpolation.

[0097] In this embodiment, as in the previous embodiment, regardless ofthe number of pixels in the image sensing device, the structure of theimage sensing device or the television system employed, a variety ofconstructions essentially equal in the working principle to thisembodiment can be realized by using an image sensing device that has anarbitrary number of vertically arranged pixels N three or more times thenumber of effective scanning lines M of each field and which allows thevertical mixing or culling of that number of pixels which is greater byat least one than the integral part of a quotient of N divided by M.

[0098] Next, a further embodiment of the present invention will beexplained. This embodiment differs from the previous embodiments in thatthe image sensing device have different arrangements of color filters.FIGS. 13A and 13B show arrangements of color filters in this embodiment.These color filters in both examples are arranged in vertical stripesand, regardless of the number of pixels to be vertically mixed orculled, the R, G, B primary color signals can be generated from one lineof output signals. FIG. 13A shows a color filter arrangement on theimage sensing device in which white filters (W=passing all colors) areused instead of the green (G) filters used in the previous embodiment.The R, G, B signals can be obtained by the following calculations.

[0099] R=W−Cy

[0100] G=Ye+Cy−W

[0101] B=W−Ye

[0102] When this color filter arrangement is used, a higher sensitivitycan be obtained than the color filter arrangement of the previousembodiment. FIG. 13B show a color filter arrangement that uses, in steadof complementary colors, color filters that pass the primary colors R,G, B. This color filter arrangement can directly produce the primarycolor signals, R, G, B and can provide a camera with good color purityand good color S/N.

[0103] With the above color filter arrangements, it is possible toproduce color signals corresponding to the three kinds of color filtersfrom each line of output signals at all times no matter how many pixelsare vertically mixed or culled. Therefore, color signals conforming tothe television system can be generated easily.

1. An electric camera comprising: an image sensing device with a lightreceiving surface having N vertically arranged pixels and an arbitrarynumber of pixels arranged horizontally, N being equal to or more thanthree times the number of effective scanning lines M of a display screenof a television system; a driver to drive the image sensing device tovertically mix or cull signal charges accumulated in individual pixelsof every K pixels to produce a number of lines of output signals whichcorresponds to the number of effective scanning lines M, K being atleast one of integers equal to or less than an integral part of aquotient of N divided by M; and a signal processing unit to generateimage signals by using the output signals of the image sensing device.2. An electric camera according to claim 1, wherein the driver drivesthe image sensing device to read out or discard the signal charges of apixel area corresponding to N−K·M pixels during a vertical blankingperiod of the television system to extract the signal charges of a pixelarea corresponding to K·M pixels from the N vertically arranged pixelsof the image sensing device during a vertical effective scanning periodof the television system and thereby obtain a number of lines of outputsignals which corresponds to the number of effective scanning lines M,N−K·M being the N vertically arranged pixels of the image sensing deviceminus a product K·M of the number of pixels to be cyclically mixed orculled K and the number of effective scanning lines M.
 3. An electriccamera according to claim 1, wherein the signal processing unit has afunction of: extracting an output signal period corresponding to thathorizontal segment which virtually matches an aspect ratio of thetelevision system with respect to a vertical height of the extractedpixels in the light receiving surface of the image sensing device, andoutputting the signals of the extracted horizontal segment over theentire horizontal effective scanning period of the television system. 4.An electric camera according to claim 1, wherein the driver drives theimage sensing device to shift, in each display cycle of the televisionsystem, positions of the pixels to be cyclically mixed or culled andthereby output interlaced signals.
 5. An electric camera according toclaim 4, wherein the signal processing unit has a function ofinterpolating vertical positions of gravity centers of the interlacedoutput signals obtained by the mixing or culling so that a phasedifference between the gravity centers on two interlaced fields is 180degrees.
 6. An electric camera according to claim 1, wherein the imagesensing device can vertically mix or cull those numbers of pixels whichare at least two of integers equal to or less than an integral part of aquotient of the number of vertically arranged pixel rows N divided bythe number of effective scanning lines M, and the driver drives theimage sensing device in at least two modes corresponding to the at leasttwo integers.
 7. An electric camera according to claim 6, wherein thedriver for the image sensing device changes the number of pixels to becyclically mixed or culled according to input information from a switch,such as a zoom switch, provided inside or outside the camera whichrequests a view angle change.
 8. An electric camera according to claim6, further including a gain adjust unit for adjusting a gain of theoutput signals of the image sensing device, wherein, when the number ofpixels to be cyclically mixed changes, a gain of the gain adjust unit ischanged according to the number of pixels to be cyclically mixed on theimage sensing device so that an output signal level of the gain adjustunit remains constant.
 9. An electric camera according to claim 1,further including an image-unstability detector for detecting animage-unstability of the electric camera, wherein a vertical height andhorizontal width size and a position of an area to be extracted from thelight receiving surface is changed according to an amount ofimage-unstability detected by the unstability detector to correct theimage-unstability.
 10. An electric camera comprising: an image sensingdevice with a light receiving surface having N vertically arrangedpixels and an arbitrary number of pixels arranged horizontally, N beingequal to or more than three times the number of effective scanning linesM of a display screen of a television system; a driver to drive theimage sensing device to vertically mix or cull signal chargesaccumulated in individual pixels of K pixels to produce, during avertical effective scanning period of the television system, a number oflines of output signals which corresponds to 1/K the number ofvertically arranged pixels N of the image sensing device, K being aninteger larger than an integral part of a quotient of N divided by M;and a signal processing unit having a function of generating imagesignals by using the output signals of the image sensing device.
 11. Anelectric camera comprising: an image sensing device with a lightreceiving surface having N vertically arranged pixels and an arbitrarynumber of pixels arranged horizontally, N being equal to or more thanthree times the number of effective scanning lines M of a display screenof a television system; a first driver to drive the image sensing deviceto vertically mix or cull signal charges accumulated in individualpixels of every K pixels to produce a number of lines of output signalswhich corresponds to the number of effective scanning lines M, K beingat least one of integers equal to or less than an integral part of aquotient of N divided by M; a second driver to drive the image sensingdevice to vertically mix or cull signal charges accumulated in.individual pixels of every K pixels to produce, during a verticaleffective scanning period of the television system, a number of lines ofoutput signals which corresponds to 1/K the number of verticallyarranged pixels N of the image sensing device, K being an integer largerthan an integral part of a quotient of N divided by M; and a signalprocessing unit to generate image signals by using the output signals ofthe image sensing device; wherein the driving by the first driver andthe driving by the second driver are selectively switched according toinput information from a switch provided inside or outside the electriccamera.
 12. An electric camera according to claim 1, further including atrigger device such as a shutter button, wherein, when a trigger isproduced by the trigger device, the signal charges accumulated inindividual pixels of the image sensing device are not cyclically mixedbut are read out independently for all pixels.
 13. An electric cameraaccording to claim 1, wherein color filters that pass first, second andthird colors respectively are arranged to cyclically appear horizontallyat three-pixel intervals and color filters that pass the same colors arearranged vertically.
 14. An electric camera according to claim 13,wherein the first, second and third colors are yellow, green and cyan,respectively.
 15. An electric camera according to claim 13, wherein thefirst, second and third colors are yellow, white and cyan, respectively.16. An electric camera according to claim 13, wherein the first, secondand third colors are red, green and blue, respectively.